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METHOD OF MOTION ESTIMATION FOR TRANSMISSION 
COST REDUCTION OF MOTION VECTORS 

The invention relates to a method of movement 
5 estimation applied to MPEG-type video coding. 

The majority of movement-estimation algorithms 
implemented in video coding use the technique of "block 
matching" . 

The image is segmented into blocks of size N*N, 
10 called macroblocks, and the estimator searches for the 
vector minimizing the difference between a block of the 
current image and a block of the reference image. This 
difference is generally an MSE (Mean Square Difference) 
or MAE (Mean Absolute Difference) calculated on the 
15 luminance pixels. 

This type of estimator can supply a heteroge- 
neous movement field since it is based on the varia- 
tions of luminance and not on the actual movement in 
the sequence. This may entail an overhead for the cod- 
20 ing of the vectors by the coder, the coding generally 
being of differential type, and thus a reduction in 
performance . 

The object of the invention is to remedy the 
abovementioned drawbacks . 

25 Its subject is a method of movement estimation 

including segmentation of the video image into image 
blocks, movement estimation per image block in order to 
obtain a movement vector field, characterized in that 
it includes a stage of reassignment of a vector to a 

30 block by selecting one movement vector from among N 
predominant vectors belonging to the vector field. 

According to one particular implementation, for 
a predominant vector, second-order regional maxima are 
detected so as not to be taken into account during the 

35 selection of the other predominant vectors. 

According to another implementation, the pre- 
dominant vectors are selected in each of the four 
directions. 
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According to a particular implementation of the 
method, the selection of the reassigned vector is based 
on the value of the inter-displaced-image difference 
(DFD) . 

5 A particular characteristic of the invention 

consists in adopting the zero vector if the DFDs asso- 
ciated with the N predominant vectors are greater than 
the DFD associated with the original vector, or in 
actually keeping the original vector if the DFDs asso- 

10 ciated with the N predominant vectors are greater than 
the weighted DFD associated with the original vector. 

According to another implementation of the 
method, the selection of the reassigned vector is based 
on the calculation of the activity (spatial gradient) 

15 in the inter-image difference block (current block - 
estimated block) . If the activities corresponding to 
the N predominant vectors are greater than the activity 
corresponding to the original vector, the zero vector 
is adopted* If the activities corresponding to the N 

20 predominant vectors are greater than the weighted 
activity corresponding to the original vector, the 
original vector is kept. 

According to another particular implementation 
of the method, for each image, the predominant vectors 

25 are chosen from among the field of vectors of the cur- 
rent image and the field of vectors of at least one 
preceding image. 

By virtue of the invention, the movement vector 
fields calculated by an estimator of the "block match- 

30 ing" type can be homogenized. 

The characteristics and advantages of the 
invention will emerge better from the following de- 
scription, given by way of example and by reference to 
the attached figures, in which: 

35 - Figure 1 represents a histogram of the move- 

ment vectors, 

- Figure 2 represents a regional-maxima search 

window. 
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- Figure 3 represents an example of median fil- 
tering, 

- Figure 4 represents an example of the preced- 
ing image vectors being taken into account, 

5 - Figure 5 represents movement-vector fields 

during a zoom, 

- Figure 6 represents various types of movement 
which may be detected. 

The homogenization of the vector field is 
10 obtained via a method of conditional reassignment. 

The vectors, associated with the images of a 
sequence, are calculated and stored by the estimator. 

In order to carry out processing on the vec- 
tors, a two-dimensional histogram is constructed with 
15 dimensions of 512*512 in which the coordinates repre- 
sent the values (dx, dy) which are the values of the 
horizontal and vertical components of these vectors. 

Figure 1 represents, on the left-hand part, an 
image consisting of macroblocks to which the movement 
20 vectors are allocated and, on the right-hand part, the 
corresponding histogram. 
Choice of predominant vectors 

In order to make the movement field more homo- 
geneous, the idea is to adopt a certain number of vec- 
25 tors, which is fixed in the first place by the user. 
This number will be larger in proportion to the hetero- 
geneity of the movements. 

The first solution consists in adopting the N 
vectors corresponding to the highest frequencies of 
30 appearance. 

Another possibility is to stipulate that the 
algorithm choose N/4 predominant vectors in each of the 
four orientation planes. This solution can be adopted 
as an option, as an output criterion upon detection of 
35 zoom in the sequence.. This is because such a phenomenon 
entails distribution in all directions of the vector 
field. 

The last solution envisaged is to carry out 
detection of the regional maxima. This is because the 
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problem, in the first solution, is that it is possible 
to have several contiguous maxima, which do not confer 
enormous advantages compared with the fact of adopting 
fewer of them. 

5 The histogram is therefore scanned, rejecting 

those vectors among the N predominant vectors appearing 
in the vicinity of other more predominant vectors. Thus 
the existence of these second-order maxima is identi- 
fied by looking at the histogram to see whether two 
10 maxima lie in the same window, for example with dimen- 
sions 3*3. 

Figure 2 represents such a window, referenced 
1, for searching for regional maxima, this window being 
centred around the predominant vector adopted (dX, dY) , 

15 the number of occurrences of which is n. 

Choice of the vector allocated to a macroblock MB. Re- 
assignment 
- Method of the DFD 

Once the predominant vectors have been 

20 extracted, a criterion remains to be found for reas- 
signing each of these vectors to each MB. Since the 
movement estimator uses the criterion of the minimum 
DFD { Displaced-Frame Difference) to calculate the move- 
ment vectors, it seems useful to use this criterion to 

25 find the best possible correspondence between the vec- 
tors adopted and the macroblocks of the image to be 
processed. 

After ordering the vectors in increasing order 
of their frequency of appearance, the calculation of 
30 DFD associated with each of these vectors is carried 
out for each MB. This calculation can be expressed sim- 
ply by the following formula: 

N-l N-1 

Dfd(i,j)^ ^ ^ I MBCurrent (i-hkrj+1) -MBReference (i-i-k-hdy^ j-hl-hdx) \ 

k = 0 1 = 0 



35 in which (i, j) are the coordinates of the 

MB to be processed; 

N (= 16) is the size of the MB; 
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(dx, dy) are the components of the 
vector to be tested, belonging to [-128; +127.5]. 

It is important, before applying this formula, 
to check that the vector to be tested does not point 
outside the reference image. If no vector is suitable, 
then the zero vector is assigned. 

Hence the vector corresponding to the minimum 
DFD is assigned to each MB. 

- Gradient method 

This consists in seeking, for each MB of the 
"difference" image consisting of the predicted refer- 
ence image and of the current image, the vector corre- 
sponding to the minimum gradient which gives informa- 
tion on the local activity of the MB (of horizontal and 
vertical gradient type) • 

MB_gradient= ^ block _ active 

41 lima 
blocks 



with : 



block active=MAX 



^i = 6, j = 7 
MAX 



x(i, j) - x(i + I, j) 



MAX x(i, j) - x(i, j + I) 
i/j = 0 ' 



Enhancement of the reassignment 
DFD/Gradient criterion 

In order to keep certain movements, relating to 
objects of small size, the following criterion is 
defined: 

If, after application of the DFD method, the 
vector adopted for an MB generates a DFD greater than 
the weighted original DFD, the original vector is kept. 

Likewise, regarding the method of the gradient, 
for each MB obtained after inter-image difference, the 
gradient obtained by reassignment is compared with the 
gradient of the original vector. If the weighted origi- 
nal- gradient is less than the new gradient, the origi- 
nal vector is kept. 
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Filtering applied to the movement vectors 

In order to make the vector fields more homoge- 
neous, other criteria may be used, namely spatial or 
temporal filtering . 

- Spatial filtering 

The filter adopted is the two-dimensional 3*3 
median filter: 

the principle is explained below in the light 
of Figure 3 which represents an image referenced 2 
before filtering and an image referenced 3 after fil- 
tering. The vector referenced 4 is the vector- to be 
processed. 

The vertical and horizontal neighbours of the 
components of the MB in question are ordered along each 
direction (dx, dy) , then the median value of each com- 
ponent is taken. Next the various DFDs associated with 
each MB are compared, in the case in which either one 
component is filtered, or both, or no component is fil- 
tered. Hence the vector corresponding to the minimum 
DFD is chosen, the original DFD, obviously, being 
weighted, 

- Temporal filtering 

The idea of temporal coherence is to take 
account, in the reassignment of the vectors of an im- 
age, of the movement fields of the preceding images; 
this is done with a view to limiting the disparity in 
the movements from one image to another. 

To begin with, we will detail the principle of 
temporal filtering of Forward vectors (deferred- 
movement vectors) . 

Spatio-temporal histogram of Forward vectors: 

In order to take account of the various histo- 
grams, scaling of the vectors is carried out at a first 
stage, then weighting of the occurrences which is a- 
funct-ion of the position of the various histograms with 
respect to the histogram processed. 

Hence, for the P image of Figure 4, it is pos- 
sible to add to the histogram of original vectors, the 
occurrences of which have been weighted by a factor 3, 
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the occurrences of the vectors of the first B (the 
amplitude of which has been multiplied by 3) which are 
weighted by a factor 1 as well as the occurrences of 
the vectors of the second B (the amplitude of which has 
been multiplied by 3/2) which are weighted by a factor 
2. 

Temporal coherence should be relevant when uni- 
form movements are present, and breaks in movement 
(change of scene) are not present. 

Case of Backward vectors (anticipated-movement 
vectors) * 

It would be logical to think that, if there are 
uniform "Forward" movements from one image to the next, 
they would also be present in the case of the "Back- 
ward" vectors associated with the B images. In order to 
filter the latter, it must not be forgotten that the 
Backward vectors are based on the P or the I which will 
follow the B in question. Hence, for the first B, it 
may be thought that its Backward vectors will be twice 
as large as the Backward vectors associated with the 
second B. Scaling is carried out on the vectors of the 
latter by a factor of 2, and the weighted occurrences 
will be added, in the histogram associated with the 
first B. 

Detection of uniform field 

The idea of applying the reassignment with N 
vectors on sequences with multidirectional movements 
such .as a zoom, for example, is not relevant- This is 
because, in this fairly specific case, the fact of 
adopting only N predominant vectors does not make it 
possible conveniently to process the fields consisting 
of multiple vectors. 

Figure 5 represents the image of the vectors 
during the zoom. It can easily be seen that the dispar- 
ity in the field does not allow any such uniformity. 

It is therefore decided to detect, in the first 
place, a field in which the vectors are uniformly dis- 
tributed, either unilaterally, or in all directions 
(zoom). This detection is conveyed by a standard devia- 
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tion of the first predominant vector close to the 
average standard deviation calculated from among the N 
predominant vectors. This is expressed as: 

if al < threshold^aaverage => uniform field present 
5 in which the threshold is fixed by the user 

(threshold = 1.34 for example). 

Examples relating to the types of movements which 
are successfully detected are represented in Figures 
6a, c, d, 

10 The objective is, at present, not to apply the 

algorithm when cases (c) and (d) are present. These 
cases have still to be distinguished from cases (a) and 
(b) . To do that the average values of the dx and dy 
movements are examined, from among the N adopted, and 

15 it is seen whether they are close to zero. This is be- 
cause it may be observed that the movements in a zoom 
seem to cancel out if they are added, in contrast to 
unilateral movement. A maximum difference of five pix- 
els can be set for dx, dy. 

20 Limitation on the temporal filtering 

It is useful not to have to filter the histo- 
grams temporally in the event of breaks in movement. It 
is possible: 

- to store the histogram of initial or reas- 
25 signed vectors for a P-type image; 

- at the next P-type image, P (t), the new "im- 
age" vectors are compared. If they differ too much from 
their counterparts arising from P (t - n) , the original 
vectors are kept. 

30 Choice of the Number of Predominant Vectors 

The number of vectors necessary may be decided 
automatically and dynamically, in such a way that, for 
sequences with random movements (for example a sporting 
sequence) , there are more vectors than for sequences 

35 with uniform movements ("train"). 



